Machine learning relates to systems that can learn from data, rather than merely follow explicit programmed instructions. Machine learning is employed in a range of computing tasks where designing and programming explicit rule-based algorithms is infeasible for a variety of reasons.
Supervised learning is a machine-learning task of inferring a function from labeled training data. A “teacher” provides example inputs and their desired outputs to a computing system, and the goal is to learn a general rule that maps inputs to outputs. Supervised learning algorithms are commonly described as performing the task of searching through a hypothesis space to find a suitable hypothesis that will make good predictions with a particular problem. Even if the hypothesis space contains hypotheses that are very well suited for a particular problem, a suitable hypothesis may be very difficult to find.
An ensemble is a supervised learning algorithm because it can be trained and used to make predictions. The trained ensemble represents a single hypothesis. Ensembles combine multiple hypotheses in the expectation of forming a better hypothesis, i.e., an ensemble is a technique for combining many weak learners in an attempt to produce a strong learner. Evaluating the prediction of an ensemble typically employs more computation than evaluating the prediction of a single model. Ensembles may be thought of as a way to compensate for poor learning algorithms by performing a lot of extra computation. Fast algorithms such as decision trees are commonly used with ensembles although slower algorithms can benefit from ensemble techniques as well. Random forests are an ensemble learning method for classification and regression that operate by constructing a multitude of decision trees at training time and outputting the class that is the mode of the classes output by individual trees.